An electronic trading system generally includes a trading device in communication with an electronic exchange. The trading device receives information about a market, such as prices and quantities, from the electronic exchange. The electronic exchange receives messages, such as messages related to orders, from the trading device. The electronic exchange attempts to match quantity of an order with quantity of one or more contra-side orders.
Electronically distributed data will generally include data distributed from the electronic exchange in a real-time, or substantially real-time, data feed. For example, data related to a market for one or more tradeable objects in an electronic trading environment may be distributed or published by an electronic exchange and/or distributed to one or more subscribing entities, devices or client devices in real-time or substantially real-time. The data, also referred to as market data, may include or represent information for a market for the tradeable object.
The client devices receive the market data. The received market data may be further processed by the client devices. For example, a client device may process the market data to identify the market for the tradeable object and/or determine whether to electronically submit data in the form of an order message for the tradeable object to the electronic exchange. The market data may include price for information related to a market for one or more tradeable objects, order data related to trade orders, such as order messages, confirmation messages, and/or other types of messages, and fill data for information relating to one or more fills and/or execution of trade orders.
Orders pending execution for a tradeable object at an electronic exchange (also considered “working orders”) are generally kept or organized in an order book for the tradeable object. Orders for the tradeable object that are received by the electronic exchange are verified against orders in the order book to determine whether at least a portion of a quantity of the order can be matched against one or more contra-orders, and if so, at least a portion of the order may be matched. If the order cannot be matched in whole, the order, or the unmatched quantity of the order is placed in a queue in the order book for the tradeable object at the electronic exchange.
Circumstances arise when a current local copy of the order book and/or fill book of the electronic exchange (for example, a full set of Orders or Fills currently known or available) may be kept locally by one or more components of a trading system, such as at a client device. When the client device gets the current order book and/or fill book, the client device will generally download the currently active orders and/or the historical set of fills. This download occurs while the exchange system is also actively delivering or publishing real-time data related to current market data, such as current orders or fills or updates thereto. When the download stream originates from a different end-point, such as different machines, sockets, threads, message topics, and the like, the client device that is downloading the data may need to merge the downloaded data with the data received in a streaming real-time data feed.
The merging of the real-time data with the downloaded data may be complicated, may consume excessive processing resources, and may introduce delays and inaccuracies. For example, in an existing system, real-time data will arrive at the client device as the client device is also downloading already known data. In some circumstances, the real-time data will be held in a client device cache so that the real-time data can be reconciled with the downloaded items at the end of the download. This increases complexity, state management and memory requirements. Other circumstances first download all of the data and then subscribe/listen to real-time information upon download completion. This system may result in a gap or window of time between when a download of data ends and receiving the real-time data begins, where real-time items can easily be missed by the client device. As such, the client device may need to expend resources to manage the race conditions that happen at the gap. These designs will attempt to best-guess the overlap period and do a supplemental download. The race conditions and timing problems with this can be many. Existing systems that merge downloaded data with real-time data streaming information may first download a finite number of items and then switch over to receiving real-time data via the same connection. Existing systems that use multiple connections require buffering and/or caching of real-time data during a download phase, requiring a post download merge step.
As such, there is a need for merging downloaded data with one or more real-time data feeds.
Embodiments and features for merging data downloads with one or more real-time data feeds will be better understood when read in conjunction with the provided figures, which illustrate examples. It should be understood, however, that the embodiments for merging data downloads with real-time data feeds are not limited to the arrangements and instrumentality shown in the attached figures.